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ABSTRACT 

j , A random sample of 225 grade one children were given 

the Canadian Cognitive Abilities Test (CCAT) on two occasions within 
seven months. During the second adrninistration the Wise was also 
administered. KR--20 reliabilities of the CCAT for the two 
administrations were respectively .83 and ,74, The test^retest 
reiiability was ^75* The correlation coefficient of the Wise iQ's arid 
the CCAT second administration IQ's was ,63, The efficiency and 
effectiveness indices of the CCAT were also obtained, a fac ur 
analysis of the WISC and CCAT subtests indicated that the two 
instruments measured different constructs. Other useful item data 
were also obtained. (Author) 
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Abstract 

A random saraple of 225 grade one children were given the Canadian 
Cognitive Abilities Test (CCAT) on two occasions within seven months* 
During the second administration the UISC was also administered, 

KR-20 reliabilities of the CCAT for the two administrations were 
respectively ,83 and ,74, The test - retest reliability was ,75, 
The correlation coefficient of the WISC IQ's and the CCAT second 
administration IQ's was ,63, The efficiency and effectiveness indices 
of the CCAT were also obtained* 

A factor analysis of the WISC and CCAT subtests indicated that 
the two instruments measured different constructs. Other useful Item 
data were also obtained* 



FACTORIAL STRUCTURE, RELIABILlTYt VALIDITY^ 
H'FECTIVENESS AND EFFICISJCY OF THE 
CANADIAN COGNITIVE ABILITIES TEST ^ 
Blkkar S* Randhawa, Dennis Hunt 
and 

Shirley A, Aawlyk 
University of Saskatchewan 
Saskatoon, Canada 
The Cognitive Abilities Test (CAT) is a downward eKtension of 
the Lorge'Thorndike test of intelligence for use with Kindergarten 
to Grade Three* The Canadian Cognitive Abilities Test CCCAT) is 
essentially the CAT version^ but standardised on Canadian samples in 
Kindergarten and Grades One to Four classes. The CCAT normative 
samples at each level were substantially large, but vei^ little 
Inforroation is provided on the regional representation of the classes. 

There is absolucely no inforMtion provided in the test manual 
on the validity of this test except for the construct validity 
inferred from a factor analysis* It is reported in the manual that 
the factor analysis was "based on a raatched and representative sample 
of 300 pupils Cp. 26)" in grade 3 all of whom had taken the CCAT and 
the Canadian Lorge-Thorndike Intelligence Tests 5 Multi-Level Edition, 
Level A (CLTIT)» However^ no inforMtion is given on the matching 
procedure or the particular factor analytic method used# The reported 
results of the factor analysis were the unrotated factor loadings on 
tturea factors p comnunalitiesi eigen values , and cumulative percentage 
of variance accounted for. From these results the authors concludCp 



1 Paper presented at the Joint rieetlng of the AERA and NCME, New Orl^ns 
February, 1973 • 



"It is quite evident from the first factor that b^isically what 
Is being measured by all three batteries the CCAT and the Canadian 
Lorge^Thorndike Verbal and Nonverbal Batteries is a general 
reasoning factor (p* 27), Factor II is tentatively designated a 
■verbal' factor* However, the loadings o£ factor II on the four 
subsets of the CCAT are negative and on the verbal battery of the 
CLTIT are positive* This would suggest that the CCAT does not 
essentially measure the same thing as the verbal battety of the 
CLTIT, Furthermore, it is almost impossible to iMke any parsimonious 
interpretation from the matrix of unrotated factor loadings. 

The present study was designed to eKamine the construct validity ^ 
reliability, item difficulties, discrimination indices, effective- 
ness and efficiency of the CCAT test, 

« 

Method 

Sample 

A v/andom sampZe of 225 children was drawn from a total of 
914 children registered in a Canadian mid-western school Jurisdiction. 
Procedure 

All the children in the sample had taken the CCAT Briery I, 
Form I in the seventh month of the grade one program^ Approximately 
seven months after thj first administrations these children were 
retested with the CCAT Primary I, Form I and at the same time the 
Wise was administered. The mean chronological age of the present 
sample at the time of the post-test was 87.2 months. The Standard- 
isation Year II sample mean chronological age was 94*5 months.^ 
Since the mean chronological age difference was significant, the 
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authors felt justlflad in using the CCAT Pl/Fl for the post-test. 
Data Analysis 

KR-20 reliability coefficients of the CCAT for Che first and 
the second administration were obtained. The stability coefficient 
between the two administrations was also calculated. 

To examine the .struct validity of the CCAT an intercorrelation 
oatriK of the four CCAT subscorea; Oral Vocabulary <0V) , Relational 
Concepts CRC), Multi-mental (MM) ^ and Quantitative Concepts (QC) and 
the ten WISC subscorea; General Information (D^ General Comprehen* 
alon (CO), Arithmetic (A), Similarities <S) ^ Vocabula^ (V), Picture 
Complation (PC) ^ Picture Arrangement (PA)^ Eloek Design (BD), Object 
Assembly (OA) and Coding (C) was obtained and subjected to a Principal 
component factor analysis* Four principle corapDnents, corresponding 
to eigenvalues greater than one^ were retained, and the resulting 
factor matrix rotated to a varimaK criterion. 

The proportion of Sb who answered each item correctly was 
calculated* This was used bh an estimate of item difficulty* The 
point-biaerial correlation coefficients between the item and the total 
gcores were obtained as eatimates of discrimination indices* 

The efficiency and the eff ectivenesa of the CCAT were determined 
using the WISC as the criterion instrument* The efficiency of a 
screening instrument is defined as the percentage of children of given 
charaeteriaticss as determined by the criterion instrument , compared 
with the total nimiber identified by the screening instrument. The 
effectiveness of an instrument is defined as the percentage of all 
the children with a given charicteristie which the instrument locates 
as compared with the total nimber of children having this character- 
istic as determned by the criterion Initrument (Pegnate & Birchp 1959). 



Results and Dlscussibn 
KR-20 reliability coefficients of the test for the first and 
second administration were respectively 0,83 and 0,74, The stabilit 
coafficient was 0,75. The criterion-related validity coefficient 
based on the corf elation between the WlSC IQ and the CCAT second 
administration IQ scoras was 0,53# However ^ the correlations of 
the CCAT IQ scores with the WISC verbal and performance IQ scores 
were respectively 0*55 and 0,57, 

Since no parsimonious Interpretation was possible from the 
unrotated factor loadings reported in the CCAT^ Pl/Flj Examiner* s 
Manual (1970, p, 26), these loadings were rotated to varlmaK 
criterion,' The resulting varlmaK factor matrix is presented in 
Table 1# It is evident from these results ^ that unlike the one 

Insert Table 1 about here 

general reasoning factor inferred from the unrotated factor loadings 
by the authors of the CCAT, there are three distinct interpretable 
factors. Factor I is a verbal factor and factor III Is a non-verbal 
factor. It is interesting to note that factor II , on which the CCAT 
subscores load significantly , is typical of the CCAT only* Thus it 
appears that whatever the CCAT is measuring, it is independent of 
what is being measured by the verbal and the nonverbal batteries of 
the CLTIT* Therefore an examination of the unrotated factor matrlK 
is Insufficient to substantiate the construct validity of the test. 

The unrotated and the varlmaK factor loadings of the WISC and 
the CC^T subscores are given in Table 2, Again an eKaminatlon of | the 



Insert Table 2 about here 

unrotatad factor loadings could erroneously be interpretted as one 
general factor* This is almost always possible in factoring in 
common factor space using principal components , because the first 
eigenvalue is the largest in the defined space. Ilcweverj the variiMK 
rotation of the above raatriK gives four interpretable and parsimonious 
factors* Among these four factors, one factor (III) again is typical 
of the CCAT only* The other three factors can be labelled verbal ^ 
reasonings and nonverbal respectively* Thus it is evident that the 
CCAT is measuring something which is independent of that which is 
being measured by the WISC* 

Item difficulties and discrimination indices of the CCAT items 
for the present sample are given in Tables 3 and 4* Since no data 
were available in the Mnual on these indices , it was felt necessary 

Insert Tables 3 and 4 about here 



to enamine these indices to evaluate the items of the CCAT, As can 
be seen from Tables 3 and 4j there are many items in each subtest 
whlah are very easy and have no discriminating characteristics* On 
the other hand I some itemsj such as item 22 in the Relational Concepts 
subtest, are very difficult and have again very low discrimination 
indices. Lack of discriminating power of the various items of the 
CCAT could be attributed to the use of Primary I instead of Primary II. 
However j aa pointed out earlier, the present aaraple was, on the averagep 
seven monthi younger than the standardization sample and the post-test 



was administered in the first two months of Grade 2, In any case 
there appears to be a need for more technical data in order to aid 
the selection of the appropriate level of the CCAT* 

The efficiency and the effectiveness of the CCAT against the 
tiTISC as the criterion instrument for the low and high criterion scores 
are given :ln Table 5* Both the efficiency and the effectiveness are 

Insert Table 5 about here 

low for this sample* Thus if the CCAT is to be used to identify low 
and high IQ subjects for referrals > it should be used with caution. 
It was found for the present sample that to ensure 100% effectiveness 
the CCAT IQ cutoff points should be 105 and 116 respectively, 

Frc the results of the present study, it would appear that 
the CCAT requires further refinement ^ particularly in the lower 
levels* If the CCAT is going to be used in Canada ^ it is essential 
that its users be assured of its validity and reliability* In an 
era of distrust of intelligence testing and public concern for misuse 
of intelligence scores , it is extremely Important that the measuring 
Instri^ents used are as free from technical deficiencies as is possible* 
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1 Paper presented at the joint meeting of the AERA and NCME, New Orleans, 
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